Administrators of enterprise software systems face a particularly troublesome problem in attempting to deploy new applications (e.g. to ensure license compliance) to all the computer devices in their care. For simplicity, the devices to which it may be necessary to deploy new software, will be known henceforth as “recipient devices”. This problem is proving particularly difficult in view of the ever-increasing complexity of today's IT environments and the growing use of pervasive devices (e.g. PDAs). There are a number of enterprise software distribution programs currently available (e.g. IBM's Tivoli Software Distribution program) which enable administrators to install, configure, and update software on networked systems from a single source system, thereby eliminating the need for manually updating software. These software distribution programs can implement “push” (i.e. a reverse client/server [or server-initiated]) or “pull” (client initiated) procedures to identify non-compliant recipient devices and transmit messages thereto to cause disseminated agents on the recipient devices to execute an installation code and thereby update their software. For simplicity, software distribution programs that implement a push procedure or a pull procedure will be known as a “push software distribution program” or a “pull software distribution program” respectively.
In the on-going pursuit of more reliable software, one of the issues that faces software designers, is that of assessing the scalability of the software. More particularly, for a software distribution program, the scalability issue becomes one of determining how the program would react when the number of recipient devices addressable by the tool significantly increases.
Scalability testing can be performed as a series of load tests with different “real-world” hardware or software configurations (henceforth known as “real targets”). Scalability testing may also involve simulating conditions in the application under test. These simulated conditions will be known henceforth as “virtual targets”. The targets for a software distribution program, could include a large number (i.e. 50000-100000) of recipient devices to be addressed by the program.
There are a number of load/stress testing programs currently available (“An Overview of Load Test Tools”, J. Buret and N. Droze, http://clif.objectweb.org/load_tools_overview.pdf) that produce simulated targets (e.g. simulated management requests from several remote clients) for pull software distribution programs. However, none of these load/stress testing programs can simulate targets for push software distribution programs.
In order to evaluate how a push software distribution program would react to a huge number of recipient devices it is usually necessary to simulate the recipient devices (e.g. by starting several processes on the same machine, wherein these processes emulate the target recipient). However, this can be problematic when the footprint of each target (recipient device) is very large, from a memory and/or CPU utilization perspective. In this case, only a small number of recipient devices can be emulated, which reduces the effectiveness of the scalability test.
Similarly, targets may be “virtual” entities wherein an actual operation should be executed by another entity. This may occur, for example, when a request is for a pristine operation (e.g. a base-metal OS installation) to a list of machines that have to be rebuilt from scratch. In this case, the management server is notified only when the reinstallation operation is completed for each target machine (i.e. not when the operation has been started at a pristine server). Thus, there is a set of virtual targets (the machines that have to be re-pristined) and one real physical target (the pristine server) that manages, controls and monitors the execution of the operation on the targets.
While the description so far, has focussed on the problem of testing the scalability of a system for distributing software (or software updates) to a large number of recipient devices, it will be appreciated that the scalability testing problem also applies to any other “reverse client-server” systems management application, where a single node issues requests to a consistent set of target recipient devices to execute a systems management action (e.g. installation of software, a general lifecycle operation on software (e.g. commit, remove, upgrade), hardware or software inventory operation, distributed job scheduling, distributed resource monitoring, etc.).